IEEE Transactions on Neural Systems and Rehabilitation Engineering

There are many alternative methods to joystick control for control of Electric Powered Wheelchairs for users with neuromuscular disabilities, such as muscular dystrophy, and spinal cord injuries, such as tetraplegia. However, these methods- which include the sip-and-puff method, head and neck movement, blinking, or tongue movement- hinder social interaction, and are therefore detrimental to user independence. In recent years, research has explored the use of Electromyography (EMG) signals from alternative muscles to control a powered wheelchair, consequently increasing the quality of life of these users. The Auricular Muscles (AM) may be suitable, as they are controlled separately from the facial nerve and are vestigial in humans, making them advantageous for powered wheelchair control for users with tetraplegia. Additionally, they are located around the ear, adding a level of cosmesis when designing wearable sensors and prosthesis. This paper extracts and implements two control strategies from current literature and, for the first time, compares them directly, demonstrating viable implementation approaches for an online EMG-based powered-wheelchair control system. A Support Vector Machine (SVM) was developed and various window lengths were compared, with the most accuracy and real-time effectiveness found at 300ms. A study with three participants demonstrates the feasibility of these methods of control as well as experimental results to guide the potential AM use.

Following stroke, injury, or exposure to physically limiting conditions, limbs can become physiologically compromised. In particular, motor and fine-dexterity tasks involving the arm, particularly in locomotion, grasp and release, can be influenced becoming either delayed and having to deal with greater force demands. Current prosthetic systems use electromyography (EMG)-based techniques for creating functional sensorimotor platforms. However, several limitations in practical use and signal detection have been identified in these systems. Accelerometer-based sensorimotor systems have been suggested to overcome these limitations but only proof-of-concept has been demonstrated. Here, we explore design specifications for accelerometers being developed for prosthetic integration. We have developed optimizations for the current model, evaluated system properties to enhance sensitivity and reduce signal noise, and performed a pilot test using simulation to test this model. The data suggest these novel design parameters can enhance signal detection, when compared to conventional accelerometers. Future avenues should focus on validation of this design prototype in a full prosthetic system.

Electromyography (EMG) is an electrical voltage potential linked to muscle contraction, resulting in human joint motion, such as knee flexion. Knee injuries, such as knee osteoarthritis (KOA), disrupt functional mobility of the knee joint and subsequently atrophy the muscles controlling knee movement during activities of daily living (ADL). Consequently, weakened muscles exhibiting deteriorated EMG signal fidelity are hypothesized to have discernible signal patterns from a healthy individuals EMG signals. Pattern recognition algorithms are useful for mapping a set of complex inputs (EMG signals and knee angles) to classify knee health status (injured vs. healthy). A secondary outcome is to predict future knee angles from previous input signals to inform a robotic knee exoskeleton to apply real-time torque assistance to a patient during ADL. A Decision Tree Classifier, Random Forest, Naive Bayes, and a Feed Forward Neural Network (Fully Connected) were used for binary classification (healthy vs. injured). Partial Least Squares Regression, Decision Tree Regressor, and XGBoost were used to predict future joint angles for the regression task (knee angle prediction). Overall, the Random Forest Classifier had the best overall classification performance. XGBoost and Decision Tree Regression performed the best among regression algorithms for predicting real-time angles during walking while Partial Least Squares Regression performed the best during the standing tasks. In summary, our Machine Learning methods are useful for assisting clinicians and patients during physical rehabilitation by providing quantitative insight into the patients neuromuscular control of the knee.

After stroke, paretic arm muscles are constantly exposed to abnormal neural drive from the injured brain. As such, hypertonia, broadly defined as an increase in muscle tone, is prevalent especially in distal muscles, which impairs daily function or in long-term leads to a flexed resting posture in the wrist and fingers. However, there currently is no quantitative measure that can reliably track how hypertonia is expressed on daily basis. In this study, we propose a novel time-based surface electromyography (sEMG) measure that can overcome the limitations of the coarse clinical scales often measured in functionally irrelevant context and the magnitude-based sEMG measures that suffer from signal non-stationarity. We postulated that the key to robust quantification of hypertonia is to capture the "true" baseline in sEMG for each measurement session, by which we can define the relative duration of activity over a short time segment continuously tracked in a sliding window fashion. We validate that the proposed measure of sEMG active duration is robust across parameter choices (e.g., sampling rate, window length, threshold criteria), robust against typical noise sources present in paretic muscles (e.g., low signal-to-noise ratio, sporadic motor unit action potentials), and reliable across measurements (e.g., sensors, trials, and days), while providing a continuum of scale over the full magnitude range for each session. Furthermore, sEMG active duration could well characterize the clinically observed differences in hypertonia expressed across different muscles and impairment levels. The proposed measure can be used for continuous and quantitative monitoring of hypertonia during activities of daily living while at home, which will allow for the study of the practical effect of pharmacological and/or physical interventions that try to combat its presence.

Haptic feedback can play a useful role in rehabilitation and brain-computer interface applications by providing users with information about their system or performance. One challenge delivering tactile stimulation is not knowing how the haptic sensation is actually perceived, irrespective of the stimulation amplitude, during real-world use and beyond controlled psychophysical experiments. In a participant with chronically implanted electrocorticography arrays, we observed that perceived intensity of haptic vibration on the fingertips was represented in the high-gamma (HG) frequency band (70-170 Hz) in the somatosensory cortex. The five fingers of the participants right hand were represented by distinct channels in the implanted array and modulated by the vibration amplitude at the fingertips. Although it reliably varied with the vibration amplitude, we found that HG activity had a stronger relationship with the actual perceived intensity of haptic stimulation (rs= 0.45, p < 10-6). These results demonstrate that neural signals, specifically HG activity, in the somatosensory cortex can represent qualities of perceived haptic intensity regardless of the stimulation amplitude, which could enable a new way to passively quantify or ensure effective haptic feedback to a user.

In this paper an automated data labeling (ADL) neural network was proposed to streamline dataset collecting for real-time predicting the continuous motion of hand and wrist, these gestures are only decoded from a surface electromyography (sEMG) array of eight channels. Unlike collecting both the bio-signals and hand motion signals as samples and labels in supervised learning, this algorithm only collects the unlabeled sEMG into an unsupervised neural network, in which the hand motion labels are auto-generated. The coefficient of determination (r2) for three DOFs, i.e. wrist flex/extension, wrist pro/supination, hand open/close, was 0.86, and 0.87 respectively. The comparison between real motion labels and auto-generated labels shows that the latter has earlier response than former. The results of Fitts law test indicate that ADL has capability of controlling multi-DOFs simultaneously even though the training set only contains sEMG data from single DOF gesture. Moreover, no more hand motion measurement needed which greatly helps upper-limb amputee imagine the gesture of residual limb to control a dexterous prosthesis.

AO_SCPLOWBSTRACTC_SCPLOWShared human-robot control for assistive machines can improve the independence of individuals with motor impairments. Monitoring elevated levels of workload can enable the assistive autonomy to adjust the control-sharing in an assist-as-needed way, to achieve a balance between user fatigue, stress and independent control. In this work, we aim to investigate how heart-rate variability features can be utilized to monitor elevated levels of mental workload while operating a powered wheelchair, and how that utilization might vary under different control interfaces. To that end, we conducted a 22 person study with three commercial interfaces. Our results show that the validity and reliability of using the ultra-short-term heart-rate variability features as predictors for workload indeed are affected by the type of interface in use.

There has been an accelerated surge to utilize the deep neural network for decoding central and peripheral activations of the humans nervous system to boost up the spatiotemporal resolution of neural interfaces used in neurorobotics. Such algorithmic solutions are motivated for use in human-centered robotic systems, such as neurorehabilitation, prosthetics, and exoskeletons. These methods are proved to achieve higher accuracy on individual data when compared with the conventional machine learning methods but are also challenged by their assumption of having access to massive training samples. ObjectiveIn this letter, we propose Dilated Efficient CapsNet to improve the predictive performance when the available individual data is very minimum and not enough to train an individualized network for controlling a personalized robotic system. MethodWe proposed the concept of transfer learning using a new design of the dilated efficient capsular neural network to relax the need of having access to massive individual data and utilize the field knowledge which can be learned from a group of participants. In addition, instead of using complete sEMG signals, we only use the transient phase, reducing the volume of training samples to 20% of the original and maximizing the agility. ResultsIn experiments, we validate our model performance with various amounts of injected personalized training data (25%-100% of transient phase) that is segmented once by time and once by repetition. The results of this paper support the use of transfer learning using a dilated capsular neural network and show that with the use of such a model, the knowledge domain learned on a small number of subjects can be utilized to minimize the need for new data of new subjects while focusing only on the transient phase of contraction (which is a challenging neural interfacing problem).

For a seamless integration of electroencephalography (EEG)-based motor imagery brain-computer interfaces (MI-BCIs), it is vital to be able to classify movements of the same joint. However, a fundamental challenge in classifying the same joint movements arises from the close spatial proximity of the corresponding brain regions. To address this challenge, we explore the feasibility of distinguishing up and down movements specific to the right upper limb using multiple frequency bands combined with a channel averaging method. Six electrodes positioned in close proximity to the motor cortex and two distinct frequency bands: mu (8-12Hz) and beta (12-30Hz) were selected. This isolates and enhances electromagnetic activity in the brain commonly associated with motor and cognitive processing. The results of our study revealed promising outcomes across two classification methods. Utilizing a support vector machine (SVM) classifier, our proposed approach achieved an average accuracy of 59.3% and a k-nearest neighbor(KNN) classifier approach yielded an average accuracy of 61.63% in distinguishing between upward and downward movements of the right arm. These results demonstrate the potential of combining spatially focused EEG acquisition with frequency-specific analysis for improved MI-BCI performance.

Brain-computer interface (BCI) can assist paralyzed patients in controlling external devices and improve their quality of life. However, existing intracortical BCIs entail risks of infection and challenges of long-term stability. In this study, we report on a tetraplegic patient implanted with our newly developed wireless minimally invasive BCI, NEO, in which eight Pt-Ir electrodes were placed epidurally over the hand area of the right sensorimotor cortex to record field potentials. We found that epidural neural signals from the hand area simultaneously represented both contralateral and ipsilateral movements. The spatio-spectral patterns of different movements exhibited prominent distinctions, revealing a bilateral representation structure of limb movements. Moreover, when two limb effectors (e.g., hand and elbow) moved simultaneously, their neural patterns exhibited non-additive changes relative to single movements. Based on these findings, we proposed a fine grained bilateral single/dual-movement decoding scheme for two-dimensional target control, thereby extending the degrees of freedom (DoF) of minimally invasive BCI systems and enhancing the information transfer rate (ITR). In two-dimensional center-out and web-grid tasks, the system achieved mean Fitts ITRs of 36.7 bpm and 30.0 bpm, respectively, with hit rates exceeding 91%. Neural recordings remained stable for over 18 months, and the decoder maintained stable performance for over 6 months without recalibration, demonstrating the safety and reliability of long-term home use.

BackgroundSpinal cord injuries significantly impair patients ability to perform daily activities independently. While cortically implanted brain-computer interfaces (BCIs) offer high communication bandwidth to assist and rehabilitate these patients, their invasiveness and long-term stability limit broader adoption. MethodsWe developed a minimally invasive BCI with 8 chronic epidural electrodes above primary sensorimotor cortex to restore hand functions of tetraplegia patients. With wireless powering and neural data transmission, this system enables real-time BCI control of hand movements and hand function rehabilitation in home use. A complete spinal cord injury (SCI) patient with paralyzed hand functions was recruited in this study. ResultsOver a 9-month period of home use, the patient achieved an average grasping detection F1-score of 0.91, and a 100% success rate in object transfer tests, with this minimally invasive BCI and a wearable exoskeleton hand. This system allowed the patient to perform eating, drinking and other daily tasks involving hand functions. Additionally, the patient showed substantial neurological recovery through consecutive BCI training, regaining the ability to hold objects without BCI. The patient exhibited a 5-point improvement in upper limb motor scores and a 27-point increase in the action research arm test (ARAT). A maximal increase of 12.7 V was observed in the peak of somatosensory evoked potential (SEP), which points to a considerable recovery in impaired spinal cord connections. Moreover, a high-frequency component (200-300 Hz) in SEP that was initially undetectable gradually emerged and became significant, indicating notable reorganization of the underlying neural circuits. ConclusionsIn a tetraplegia patient with complete spinal cord injury, an epidural minimally invasive BCI assisted the patients hand grasping to perform daily tasks, and 9-month consecutive BCI use significantly improved the hand functions.

Most individuals with tetraplegia retain some myoelectric function in their forearms, which offers the possibility of using surface electromyographic (sEMG) control for human-computer interaction (HCI). We demonstrate the potential of this approach by showing that people with motor-complete (n=5) and motor-incomplete (n=2) tetraplegia can accurately control myoelectric activity in their forearm to perform discrete button-click and continuous positioning tasks. These control inputs were mapped to the firing rate of motor units detected by a wireless wristband sensor designed for everyday use. Participants completed four testing sessions to assess their speed and accuracy. Motor units that displayed a wide dynamic range in their firing rate performed best during tasks requiring continuous, single-axis control. Interestingly, the level of impairment did not affect performance on the clicking and 1D cursor control tasks. However, those with motor-incomplete injuries showed greater independent control over two motor units than participants with motor-complete injuries, who exhibited stronger coupling between units. Participants also confirmed the practical utility of the device, successfully placing and removing the sEMG wristband on their own and consistently rating it as comfortable and easy to manage. These findings are significant because they offer the first demonstration of motor unit-based control in individuals with cervical spinal cord injury (SCI) using a fully wearable wristband interface, highlighting the feasibility of moving these systems out of the lab and into daily life. One-Sentence SummaryPeople with tetraplegia used a wristband sensor to detect forearm motor unit firing and perform human-computer interaction tasks.

Although state-of-the-art myoelectric prostheses offer persons with upper limb amputation extensive movement capabilities, users have not been afforded a reliable means to control common movements required in daily living. Many proposed prosthesis controllers use pattern recognition, a method that learns and recognizes patterns of electromyographic (EMG) signals produced by the users residual limb muscles to predict and execute device movements. Such control becomes unreliable in high limb positions--a problem known as the limb position effect. Pattern recognition often uses a classification algorithm; simple to implement, but limits user-initiated control to only one device movement at a time, at a single speed. To combat position-related control deficiencies and classification controller constraints, we developed and tested two recurrent convolutional neural network (RCNN) pattern recognition-based solutions: (1) an RCNN classification controller that uses EMG plus positional inertial measurement unit (IMU) signals to offer one-speed, sequential movement control; and (2) an RCNN regression controller that uses the same data capture technique to offer simultaneous control of multiple movements and device movement velocity. We assessed both RCNN controllers by comparing them to a commonly used linear discriminant analysis classification controller (LDA-Baseline). Participants without upper limb impairment were recruited to perform multipositional tasks while wearing a simulated prosthesis. Both RCNN classification and regression controllers showed improved functional task performance over LDA-Baseline, in 11 and 38 out of 115 metrics, respectively. This work contributes an RCNN regression-based controller that provides accurate, simultaneous, and proportional movements to EMG-based technologies including prostheses, exoskeletons, and even muscle-activated video games.

We live in a multisensory world, where all our senses work together for giving us a fulfilling experience of the environment that we are in or during our use of immersive technologies. For gaining more insight into the temporal scale understanding of the integration phenomenon EEG based BCI can give us the understanding of the transient changes in the brain. In this study, we investigated the potential of incorporating haptics into crossmodal correspondence based research to induce MSI effect through either the active touch users feedback or crossmodal correspondences with visual and auditory modalities, such as Kiki Bouba effect. We designed two experiments: O_LIVisual stimuli were presented on a standard computer monitor, and auditory stimuli were delivered through computer dynamics. Participants responded using left or right hand by pressing either CapsLock or Enter buttons respectively. Visual cue consisted of a red circle displayed randomly either on the left or on the right side of the screen. Auditory cue was a brief high tone presented through left or right headphones for 500 ms. Text stimuli that appeared on the screen instructed participants to respond with their left or right hand. Before each trial there was a fixation central cross displayed for 500 ms. C_LIO_LIThis experiment was inspired by previous studies on Kiki-Bouba correspondence. Visual stimuli consisted of 4 shapes - circle, triangle, polygon with 6 vertices, and star - presented on a computer screen. Locations of the visual stimuli were randomized. Auditory stimuli were generated using the Online Tone Generator website (https://onlinetonegenerator.com/). 2 sets of sounds were used: the first set included sine, triangle, square, and sawtooth waveforms, each at a frequency of 500 Hz; the second set included sawtooth waveforms at frequencies of 50 Hz, 300 Hz, 600 Hz, and 2000 Hz (summarised in Table 2). C_LI O_TBL View this table: org.highwire.dtl.DTLVardef@4b63fdorg.highwire.dtl.DTLVardef@191b6cborg.highwire.dtl.DTLVardef@177e6f4org.highwire.dtl.DTLVardef@dba414org.highwire.dtl.DTLVardef@1f13b3c_HPS_FORMAT_FIGEXP M_TBL O_FLOATNOTable 2:C_FLOATNO O_TABLECAPTIONVisual and auditory stimuli used in Experiment 2 C_TABLECAPTION C_TBL Results suggested that it is indeed possible to achieve this type of integration without relying on complex haptic devices. Introducing haptics into BCI technologies through feedback touch or crossmodal correspondances holds potential to improve the user experience and information transfer rate (ITR). Participants, as expected, showed the lowest reaction times in congruent sequential test and the highest - in incongruent HAV cues based test. This indicates the importance preference for sequential cue presentation over simultaneous one. The time was significantly higher in case of Incongruent Haptic cues.

Instrumented gait assessment in pediatric populations is often constrained by the complexity and lack of portability of traditional motion capture systems. In this article, we propose a deep learning approach utilizing a one-dimensional (1D) U-Net architecture to accurately estimate ankle and knee joint angles in the sagittal plane from surface electromyography (sEMG) signals. We analyzed data from the tibialis anterior and medial gastrocnemius of 25 typically developing children (ages 4-16) to evaluate the models performance and the influence of age-related gait maturation. The proposed 1D U-Net achieved high predictive accuracy for the ankle joint (RMSE: 3.6{degrees}) and the knee joint (RMSE: 4.1{circ}). Experimental results demonstrated that incorporating the toe-off event as a temporal marker significantly enhanced prediction stability during transitional gait phases. Furthermore, Statistical Parametric Mapping (SPM) was employed to identify systematic errors, which were primarily localized during initial contact and pre-swing but remained below clinically relevant thresholds. The findings reveal that prediction accuracy increases with age, reflecting more stable neuromotor patterns. This study demonstrates that a 1D U-Net can reliably decode lower-limb kinematics from sEMG alone, enabling the development of simplified, non-invasive, and portable pediatric gait assessment tools that can be integrated into the control strategies of assistive devices.

Accurate interaction force estimation can play an important role in optimization human-robot interaction in exoskeleton. In this work, we propose a novel approach for system identification of exoskeleton dynamics in presence of interaction forces as a whole multi-body system regardless of gait phase or any assumption on human-exoskeleton interaction. We hanged the exoskeleton through a linear spring and excited the exoskeleton joints with chirp commands while measuring the exoskeleton-environment interaction force. Several structures of neural networks have been trained to model the exoskeleton passive dynamics and estimate the interaction force. Our testing results indicated that a deep neural network with 250 neurons and 10 time delays can obtain sufficiently accurate estimation of the interaction force, resulting in 1.23 of RMSE on Z-normalized applied torques and 0.89 of adjusted R2.

In the field of robotic exoskeleton control, it is critical to accurately predict the intention of the user. While surface electromyography (EMG) holds the potential for such precision, current limitations arise from the absence of robust EMG-to-torque model calibration procedures and a universally accepted model. This paper introduces a practical framework for calibrating and evaluating EMG-to-torque models, accompanied by a novel nonlinear model. The framework includes an in situ procedure that involves generating calibration trajectories and subsequently evaluating them using standardized criteria. A comprehensive assessment on a dataset with 17 participants, encompassing single-joint and multi-joint conditions, suggests that the novel model outperforms the others in terms of accuracy while conserving computational efficiency. This contribution introduces an efficient model and establishes a versatile framework for EMG-to-torque model calibration and evaluation, complemented by a dataset made available. This further lays the groundwork for future advancements in EMG-based exoskeleton control and human intent detection. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.

The purpose of this study is to develop a deep neural network model for predicting soleus muscle activation patterns from time-series lower-limb joint angles collected during level ground walking at different speeds and to evaluate the prediction performance. An open dataset was used to obtain full lower-limb kinematics, including pelvis, hip, knee, and ankle joints, and soleus muscle activation patterns from 20 control adults. Long short-term memory (LSTM)-based deep learning models were developed and then evaluated for prediction performance by conducting both the random split cross-validation (CVM1) and the leave-one-subject-out cross-validation (CVM2). For both cross-validation methods, the developed models yielded promising error and regression metrics such as the root mean square error and coefficient of determination. However, the CVM2 demonstrated that the prediction performance can be sensitive to individual datasets. The subject factors, such as age, sex, and walking speed, appear to have a negligible effect on the prediction performance for the CVM1. This study demonstrated the feasibility of the developed models to be a template for a potential tool that quantifies muscle activation patterns from joint angles during level ground walking at different speeds.

When people use brain-computer interfaces (BCIs) based on event-related potentials (ERPs) over different days, they often need to repeatedly calibrate BCIs every day using ERPs acquired on the same day. This cumbersome recalibration procedure would make it difficult to use BCIs on a daily basis. We aim to address the daily calibration issue by examining across-day variation of the BCI performance and proposing a method to avoid daily calibration. To this end, we implemented a P300-based BCI system designed to control a home appliance over five days in nineteen healthy subjects. We first examined how the BCI performance varied across days with or without daily calibration. On each day, P300-based BCIs were tested using calibration-based and calibration-free decoders (CB and CF), with a CB or a CF decoder being built on the training data on each day or those on the first day, respectively. Using the CF decoder resulted in lower BCI performance on subsequent days compared to the CB decoder. Then, we developed a method to extract daily common ERP patterns from observed ERP signals using the sparse dictionary learning algorithm. We applied this method to the CF decoder and retested the BCI performance over days. Using the proposed method improved the CF decoder performance on subsequent days; the performance was closer to the level of the CB decoder, with improvement of accuracy by 2.28%, 1.93%, 1.75%, and 3.86 % on the subsequent four days, respectively, compared to the original CF decoder. The method proposed by our study may provide a novel approach to addressing the daily-calibration issue for P300-based BCIs, which is essential to implementing BCIs into daily life.

Bionic reconstruction techniques that employ surgical neuroprosthetic interfaces, biomimetic control systems, and powered mechatronics have enabled versatile and biomimetic legged gait without reliance on intrinsic gait controllers. However, relative emphasis has been placed on the emulation of sagittal plane biomechanics while neglecting to provide control over frontal plane mechanics critical for terrain adaptation. Here, we present a 2-degree-of-freedom (DOF) bionic reconstruction at the transtibial amputation level that enables continuous neural control of both sagittal and frontal ankle and subtalar joint mechanics. To demonstrate its capabilities in a case study design, we integrated a 2-DOF robotic ankle-foot device via surface electromyographic electrodes to an individual provisioned with a surgical neuroprosthetic interface that augments residual muscle afferents. The subject was able to neurally control both degrees of freedom to regain nominal gait mechanics during both level-ground walking and continuous cross-slope navigation. Furthermore, the subject strategically traversed an obstacle course by dynamically hopping between a series of discrete cross-slope blocks, adapting to the slopes, and responding to rapid foot slips. These preliminary findings suggest that bionic reconstruction techniques can restore continuous neural control over multi-DOF prostheses to achieve agile locomotion over complex terrain. One-Sentence SummaryA multi-DOF ankle-foot prosthesis under continuous neural control enables agile locomotion over complex terrain.